Gaussian Faithful Markov Trees

We study two types of graphical models in this paper: concentration and covariance graphical models. Graphical models use graphs to encode or capture the multivariate dependencies that are present in a given multivariate distribution. A concentration graph associated with a multivariate probability distribution of a given random vector is an undirected graph where each vertex represents each of the different components of the random vector, and where the absence of an edge between any pair of variables implies conditional independence between these two variables given the remaining ones. Similarly, a covariance graph reflects marginal independences in the sense that the absence of an edge between any pair of variables implies marginal independence between these two variables. These two graphical models do not only encode pairwise relationship between variables, but they also allow us to read many other conditional independence statements present in the probability distribution through separation criteria in this graph. In general, the graph may not reflect some of these conditional independence statements. When the graph encodes all of these conditional independences we say that the probability distribution is faithful to its corresponding graphical model. We present in this paper two mathematical results concerning Markov trees : graphical models corresponding to trees. Gaussian Markov trees are necessarily faithful to their concentration and covariance graphs. More formally this means that Gaussian distributions that have trees as concentration graphs are necessarily faithful (see [1]). Similarly an equivalent result can be proved for covariance graphs (see [2]). However the methods of proofs used for these two results are completely different. Graphical models are mathematical tools used to represent conditional independences in a given multivariate probability distribution (see [3,4]). Many different types of graphical models have been studied in the literature. For example, directed acyclic graphs or DAGs are commonly referred to as “Bayesian networks” (see [5]). When the graph is undirected and when such graphs are constructed using marginal independence relationships between pairs of random variables in a given random vector, these graphical models are called “covariance graph” models (see [6,7,8,9]). Covariance graph models are commonly represented by graphs with exclusively bi-directed or dashed edges (see [6]). This representation is used in order to distinguish them from the traditional and widely used concentration graph models. Concentration graphs encode conditional independence between pairs of variables given the remaining ones. Formally, consider a random vector X = (Xv, v ∈ V )′ with a probability distribution P where V is a finite set representing the random variables in X. The concentration graph associated with P is an undirected graph G = (V,E) where • V is the set of vertices. • Each vertex represents one variable in X. • E is the set of edges (between the verices in V ) constructed using the pairwise rule: for any pair (u, v) ∈ V × V , u 6= v (u, v) 6∈ E ⇐⇒ Xu⊥⊥Xv | XV \{u,v} (1) where XV \{u,v} := (Xw, w 6= u and w 6= v)′. 2 in ria -0 04 94 74 3, v er si on 1 24 J un 2 01 0 Note that (u, v) 6∈ E means that the vertices u and v are not adjacent in G. An undirected graph G0 = (V,E0) is called the covariance graph associated with the probability distribution P if the set of edges E0 is constructed as follows: (u, v) 6∈ E ⇐⇒ Xu⊥⊥Xv (2) The subscript zero is invoked for covariance graphs (i.e., G0 vs G) as the definition of covariance graphs does not involve conditional independences. Both concentration and covariance graphs are not only used to encode pairwise relationships between pairs of variables in the random vector X, but as we will see below, these graphs can be used to encode conditional independences that exist between subsets of variables of X. First we introduce some definitions: The multivariate distribution P is said to satisfy the “intersection property” if for any subsets A, B C and D of V which are pairwise disjoint,  XA⊥⊥XB | XC∪D and then XA⊥⊥XB∪C | XD XA⊥⊥XC | XB∪D (3) We will call the intersection property (see [3]) in (3) above the concentration intersection property in order to differentiate it from another property that is satisfied by P when studying covariance graph models. Let P satisfy the concentration intersection property. Then for any triplet (A,B, S) of subsets of V pairwise disjoint, if S separates A and B in the concentration graph G associated with P then the random vector XA = (Xv, v ∈ A)′ is independent of XB = (Xv, v ∈ B)′ given XS = (Xv, v ∈ S)′. This latter property is called concentration global Markov property and is formally defined as, A⊥GB | S ⇒ XA⊥⊥XB | XS. (4) In [8] it is shown that if P satisfies the following property: for any triplet (A,B, S) of subsets of V pairwise disjoint, if XA⊥⊥XB and XA⊥⊥XC then XA⊥⊥XB∪C , (5) then for any triplet (A,B, S) of subsets of V pairwise disjoint, if V \ (A∪B∪S) separates A and B in the covariance graph G0 associated with P then XA⊥⊥XB | XS. This latter property is called the covariance global Markov property and can be written formally as follows: A⊥G0B | V \ (A ∪B ∪ S) ⇒ XA⊥⊥XB | XS. (6) In parallel to the concentration graph case, property (5) will be called the covariance intersection property. Even if P satisfies both intersection properties, the covariance and concentration graphs may not be able to capture or reflect all the conditional independences present in We say that S separates A and B if any path connecting A and B in G intersects S, i.e., A⊥GB | S, and is not to be confused with stochastic independence, which is denoted by ⊥⊥ as compared to ⊥G. 3 in ria -0 04 94 74 3, v er si on 1 24 J un 2 01 0 the distribution, i.e., there may exist one or more conditional independences present in the probability distribution that does not correspond to any separation statement in either G or G0. Equivalently, a lack of a separation statement in the graph does not necessarily imply conditional dependence. On the contrary, when no other conditional independences exist in P except the ones encoded by the graph, we classify P as a faithful probability distribution to its graphical model. More precisely we say that P is concentration faithful to its concentration graph if for any triplet (A,B, S) of subsets of V pairwise disjoint, the following statement holds: S separates A and B ⇐⇒ XA⊥⊥XB | XS. (7) Similarly, P is said to be covariance faithful to its covariance graph G0 if for any triplet (A,B, S) of subsets of V pairwise disjoint, the following statement holds: V \ (A ∪B ∪ S) separates A and B ⇐⇒ XA⊥⊥XB | XS. (8) A natural question of both theoretical and applied interest in probability theory and statistics is to understand the implications of the faithfulness assumption. This assumption is fundamental since it yields a bijection between the probability distribution P and the graph G in terms of the independences that are present in the distribution. In this paper we present two results concerning the faithfulness assumption. The first result concerns covariance graphs and was proved by Malouche and Rajaratnam 2009 (see [2]). This result is formally written in Theorem 1. It was shown that when P is a multivariate Gaussian distribution, whose covariance graph is a tree, the probability distribution P is necessarily covariance faithful, i.e., such probability distributions satisfy property (8). Equivalently, the associated covariance graph G is fully able to capture all the conditional independences present in the multivariate distribution P . Theorem 1 Malouche and Rajaratnam (2009) Let XV = (Xv, v ∈ V )′ be a random vector with Gaussian distribution P = N|V |(μ,Σ). Let G0 = (V,E0) be the covariance graph associated with P . If G0 is a tree or more generally a union of connected components each of which are trees (or a union of “tree connected components”), then P is covariance faithful to G0. The proof of Theorem 1 requires among others a result proved by Jones and West 2005 (see [11]). This result gives a method that can be used to compute the covariance matrix Σ from the precision matrix K using the paths in the concentration graph G. The result can also be easily extended to show that the precision matrix K can be computed from the covariance matrix Σ using the paths in the covariance graph G0. We now state the result by Jones and West 2005. Let us first recall that we denote by P(u, v,G) the set of paths connecting the vertices u and v in an undirected graph G. Theorem 2 Jones and West (2005) (modified). Let XV = (Xv, v ∈ V )′ be a random vector with Gaussian distribution P = N|V |(μ,Σ) where Σ and K = Σ−1 are positive definite matrices. Let G = (V,E) and G0 = (V,E0) denote respectively the concentration and covariance graph associated with the probability distribution of XV . 4 in ria -0 04 94 74 3, v er si on 1 24 J un 2 01 0 For all (u, v) in V × V